BMC Medical Research Methodology

Background. Large language models (LLMs) are increasingly used to automate title-abstract screening in systematic reviews, but validation studies typically report sensitivity, specificity, or Cohen's kappa: metrics that are unstable under the low and unbalanced prevalence that characterises most reviews. No previous study has simultaneously combined a medical topic; a hybrid two-stage pipeline (rules + LLM); a three-way classification with a separately sampled uncertain class; stratified sampling with three independent human reviewers; and dual reporting of prevalence-robust coefficients (PABAK and Gwet's AC1). Objective. To validate a two-stage automated screening pipeline (a rule-based Python script followed by a reproducible LLM adjudication of uncertain records) against three independent human reviewers in a medical systematic review on high on-treatment platelet reactivity (HTPR), aspirin resistance, and carotid plaque. The primary validation target is the Stage 1 automatic-exclusion decision (670/904 records), which is the workload the pipeline is designed to take off the human reviewer. Methods. From 904 screened records (Stage 1: 131 INCLUDE, 670 EXCLUDE, 94 UNCERTAIN; Stage 2 LLM via API, temperature = 0: 11 INCLUDE, 83 EXCLUDE), a stratified sample of 185 records (50 per Stage 1 stratum + 35 Stage 2; seed = 42) was judged independently and blind to the automated labels by three physicians: E.P. (pipeline designer), D.C. and P.P.B. (both external to the thesis project). Concordance was quantified with sensitivity, specificity, PABAK (q = 3 throughout), Gwet's AC1, work saved over sampling (WSS), and number needed to read (NNR), AI versus each reviewer and between reviewers, overall and by stratum. Majority labels (2 of 3) are reported as a supplementary panel; they do not replace pairwise tables as the primary reference for sensitivity. Results. On the primary target, Stage1-EXCLUDE concordance was PABAK = AC1 = 1.000 versus E.P. (48/48) and versus D.C. (50/50), and PABAK = 0.940 / AC1 = 0.959 versus Berti (48/50 EXCLUDE, 2 UNCERTAIN). That stratum is 670/904 records (74.1%). Pairwise human agreement was highest between the two external readers (Costazza vs Berti: PABAK = 0.854, AC1 = 0.881) and lower when E.P. was one of the pair (0.703 / 0.774 vs Costazza; 0.621 / 0.702 vs Berti). AI versus each human remained poor (PABAK 0.225 / 0.205 / 0.222). Majority labels on 185 records were 146 EXCLUDE / 27 UNCERTAIN / 4 INCLUDE / 8 NO_MAJORITY. The machine labelled INCLUDE 0 of the 4 majority-INCLUDE records (74, 75, 76, 80). Overall sensitivity was 0.800 versus E.P. (8/10; false negatives 623, 712), 0.500 versus D.C. (1/2; false negative 76), and 0.833 versus Berti (5/6; false negative 76). Versus majority the false-negative count is 1 (record 76); records 623 and 712 are majority EXCLUDE. Stage1-INCLUDE sensitivity of 1.000 is tautological. WSS = 74.1% describes the exclusion stratum only. Conclusions. What this study validates, now with three independent physicians, is automatic exclusion at Stage 1. Fine classification of INCLUDE records is not validated: humans do not agree on which residual records are INCLUDE (triple INCLUDE intersection = record 74 only), Stage 2 missed most of the few human INCLUDE labels it received, and the machine recovered none of the four majority-INCLUDE records. Domain-familiarity as an explanation of INCLUDE divergence remains a hypothesis, not a measured variable.

Aims. It is unknown whether large language models (LLMs) can predict a trial's result at the design stage, from the information available when it is registered. We tested three LLMs prospectively on the Late-Breaking Science programme of the European Society of Cardiology (ESC) Congress 2026. Methods and results. Of 139 trials, 50 were selected by pre-specified criteria, and predictions were locked and registered on the Open Science Framework before the congress. Claude Fable 5, GPT-5.5 and Gemini 3.5 Flash, without web access, received each title and an investigator-compiled summary of the registered design. They gave the probability that the primary endpoint would be met, and a point estimate with an 80% prediction interval for a locked effect measure. Results were adjudicated from publications (19) or presenters' slides (31). Of 48 scorable trials, 28 (58%) were positive. The area under the receiver operating characteristic curve was 0.80 (95% CI 0.66-0.92) for Claude, 0.79 (0.66-0.91) for GPT and 0.74 (0.59-0.87) for Gemini, with accuracies of 73%, 71% and 67%. Among 26 trials whose effect was reported exactly as locked, the direction was correct in 88% of predictions and the median multiplicative error of ratio estimates was about 15%. The 80% prediction intervals contained the observed effect in 73.1% (Claude), 84.6% (GPT) and 57.7% (Gemini). Conclusion. From the registered design alone, LLMs showed statistically significant discrimination between positive and negative trials and usually predicted the direction and approximate size of the effect. Such predictions might help trial design, and further studies are needed.

BackgroundCochrane Risk of Bias 2 (RoB 2) assessment is methodologically demanding and resource intensive. Large language models may assist this process but can produce variable judgments. AimTo develop a rule-constrained LLM system for RoB 2 assessment and evaluate its temporal stability and agreement with human reviewers. MethodsRoB2-Evaluator was developed using GPT-5.2 with fixed RoB 2 guidance and a locked instruction framework. Seven RCTs were used for calibration before version freezing. The frozen system was evaluated on 10 separate RCTs, with repeat assessment after one week and comparison against consensus judgments from two independent human reviewers. ResultsTemporal agreement was 90.0% (weighted {kappa} = 0.93), and agreement with human consensus was also 90.0% (weighted {kappa} = 0.93). Methodological rationales were stable in 94.3% of repeated comparisons. Full AI-human conceptual alignment occurred in 85.7%. ConclusionRoB2-Evaluator showed encouraging post-freeze stability, human agreement, and rationale consistency, supporting larger independent evaluations.

Background: Conventional discrimination metrics, such as the area under the receiver operating characteristic curve (AUROC), do not characterize the composition of outcomes among patients flagged as high risk by clinical prediction models. This limitation is particularly relevant in critical care, where competing deaths and incomplete follow-up can complicate the interpretation of prediction alerts. We developed the Balanced Risk Integrity Index (BRII), a stratum-level metric defined as the proportion of target events among target events, competing deaths, and early-censored observations in a flagged high-risk stratum. BRII is algebraically related to positive predictive value (PPV) and is reported alongside its component decomposition to characterize the outcome composition of flagged patients. Methods: We evaluated BRII using seven acute kidney injury (AKI) prediction models developed in MIMIC-IV and externally validated in MIMIC-III and eICU-CRD (116,010 patients in total). Additional simulations examined the effects of competing deaths and early censoring on BRII and conventional model performance metrics. Results: For 48-hour AKI prediction in the top 10% risk stratum, BRII ranged from 0.90 to 0.96 in MIMIC-III, compared with less than 0.26 across all models in eICU-CRD, where 61.5% of evaluated patients were censored before the prediction horizon. In the eICU-CRD top 10% strata for the same target, 37.0% to 50.2% of patients were censored early, 12.2% to 15.5% had AKI, and 3.4% to 8.1% died before AKI. BRII and PPV ordered at least one model pair differently in 34 of 48 cohort-outcome-top-k groups. Under simulated random censoring for 48-hour AKI prediction, BRII declined by up to 0.80 while binary-label AUROC changed by less than 0.02. Conclusions: BRII provided complementary information on high-risk stratum composition, yielding different model rankings from PPV and revealing substantial differences across external validation cohorts. Interpreted alongside PPV and its component decomposition, BRII offers a standardized approach to evaluating clinical prediction alerts across models and observation environments.

Background: Large language models (LLMs) can support systematic reviews, but accurate individual outputs do not establish whether the final synthesis preserves the clinical question, accounts for statistical dependence and incorporates corrections. Objective: To develop and evaluate a framework linking LLM-assisted evidence processing to a versioned, auditable release of synthesis outputs. Methods: We used LLM agents to interpret sources and extract data. Deterministic code enforced statistical rules; investigators resolved material ambiguities and authorized release. We specified ten release properties covering evidence identities, statistical contributions and propagation of corrections. We retrospectively evaluated six integrity domains and historical failure events in one registered prognostic review, without an external comparator or held-out domain. Results: Fifty distinct root-cause events were documented, including 12 that had changed a pooled result before correction. Forty-six were resolved, and four remained disclosed limitations. The corpus comprised 454 reports, 445 studies, 441 cohort entities and 421 dependence clusters. Forty-one of 49 registered analyses were fitted, and eight retained explicit non-fitted states. All 39 source records across five principal analysis families reached a terminal source state. Two implementations within the project agreed across 1,217 numerical comparisons. All 94 file comparisons between release and publication packages were byte-identical. Two reviewers confirmed 39 principal records after seeing the same recommendations. Conclusions: This case provides a framework for inspecting synthesized evidence together with its provenance, statistical meaning and correction history. Comparative validity, generalizability and benefit in patient-centred care require independent evaluation.

Importance. Observational studies dominate evidence in cardiovascular outcomes research, yet no widely adopted reporting standard requires authors to disclose how much of an observed association is attributable to systematic error. The negative-control calibration framework of the Observational Health Data Sciences and Informatics consortium returns a binary answer: a calibrated P-value either crosses 0.05 or it does not. That answer conflates "is there a signal" with "is the signal credible." Objective. To define the Bias Attribution Fraction (BAF), a continuous estimator in [0, 1] that quantifies the fraction of an observed log relative risk that is attributable to systematic bias, together with its 95% credible interval, and to evaluate whether reporting calibrated P-value plus BAF plus interval width produces more defensible decisions than reporting the calibrated P-value alone. Design. Three-part study: (i) metric definition and interval construction; (ii) Monte Carlo simulation across 1920 conditions by 1000 repetitions = 1,920,000 simulated target estimates (a primary grid plus a sign-flipped companion grid of identical size, pooled for all analyses), with truth known by construction; (iii) two-arm beta-blocker target trial emulation in MIMIC IV, adhering to the TARGET guideline, with targeted maximum likelihood estimation (TMLE) and 12 negative-control outcomes. Setting. MIMIC IV (critical care) and MIMIC IV ECG, 2008 to 2019. Participants. Adults with a first intensive care unit stay and a recorded 12-lead ECG. Exposures. Two clinical questions are evaluated: (1) reaching at least 50% of guideline-recommended target dose of an index beta-blocker within 7 days of admission, sustained to 12 months, versus lower or no exposure; (2) receiving guideline-directed medical therapy (GDMT) with at least two of three foundational drug classes at adequate dose for 12 months versus fewer. Main Outcomes and Measures. One-year all-cause mortality. Primary analytic outputs: (i) the calibrated P-value from a negative-control empirical null distribution with 12 outcomes; (ii) BAF-hat with a Markov chain Monte Carlo 95% credible interval; (iii) a verdict drawn from a truth-calibrated lookup table mapping the BAF-hat by interval-width cell to a probability of being bias-dominated. Results. Across 1,920,000 simulated repetitions, the calibrated P-value alone yielded a misuse rate (declared usable while truth was bias-dominated) of 36.02%. Adding the BAF point estimate reduced misuse to 16.48%, and requiring the whole credible interval of BAF to lie below 0.5, clear of the bias-dominated zone, reduced misuse to 10.16%. Requiring in addition a calibrated probability of bias dominance below 15% reduced misuse to 3.02% at the cost of declaring 22.1% of estimates usable (versus 61.8% for the calibrated P-value alone). Read from the bias-dominated view, meaning that withholding a repetition is a positive call and being truly bias-dominated is a positive truth, the BAF estimator achieved an area under the receiver operating characteristic curve of 0.922, compared with 0.787 for the calibrated P-value. In MIMIC IV, question 1 produced a calibrated P-value of 0.894 and BAF-hat = 0.875 (95% credible interval 0.385 to 0.992); the apparent one-year mortality reduction of 16.9% was entirely attributable to systematic bias and the association was not usable as effect evidence. Question 2 also failed layer 1, with a calibrated P-value of 0.1026, and BAF-hat = 0.460 placed it in the "competitive, no verdict" zone. Conclusions and Relevance. Pairing the calibrated P-value with BAF and its 95% credible interval turns a binary credibility judgement into a three-channel report (signal existence, effect size, and bias fraction). Despite TARGET-adherent design, doubly robust TMLE, and 12 negative controls, both clinical questions in this study still failed to qualify as effect evidence, with question 1 demonstrating complete bias domination and BAF quantifying the systematic error that remained. The reporting standard proposed here can be applied to any observational study that includes negative controls, without requiring additional data.

BACKGROUND: Federal policy now requires sponsors to submit diversity action plans for clinical trials, yet the modifiable features of trial design associated with enrolled composition remain poorly characterized. English-language proficiency requirements are written into eligibility criteria and recorded as free text in a public registry, making them visible and, in principle, amenable to revision, but their relationship to enrolled composition has been examined only within single disease areas and in small samples of trials. We examined whether these requirements and documented language accommodation were associated with the racial and ethnic composition of enrolled participants. METHODS AND FINDINGS: We conducted a cross-sectional study of interventional trials registered on ClinicalTrials.gov with at least one US site and posted results, using a registry snapshot retrieved in October 2026. The primary cohort comprised the 44,487 trials with exclusively US sites, of which 21,924 (49.3%) reported baseline race and 17,434 (39.2%) reported baseline ethnicity with counts that reconciled with the posted denominator. Both exposures were classified from free-text eligibility criteria using regular expressions validated against independent physician review of 150 trials. For the English-language requirement classifier, sensitivity was 88.2% and specificity was 99.2%. The outcome was the trial-level proportion of enrolled participants in each category, analyzed using generalized linear models adjusted for intervention class, therapeutic area, sponsor, phase, enrollment, sites, start year, US Census region, and the population composition of the trial's site states. An English-language proficiency requirement appeared in 5,736 trials (12.9%); its prevalence rose from 7.4% of trials starting in 2010 to 26.2% in 2023 (odds ratio [OR] per decade 2.91, P<0.001), whereas documented accommodation, an eligibility statement allowing participation in a language other than English (Spanish in 99.9% of flagged trials), remained rare (788 trials, 1.8%). The requirement was associated with lower enrolled proportions of Asian (OR=0.85, 95% CI 0.77-0.93, P=0.001) and Hispanic/Latinx participants (OR=0.85, 95% CI 0.79-0.91, P<0.001), with no evidence of an association for White participants (OR=0.99, 95% CI 0.95-1.04, P=0.75). Documented accommodation was associated with a higher proportion of Hispanic/Latinx participants (OR=3.23, 95% CI 2.82-3.71, P<0.001). Geographic adjustment did not change the statistical significance of the requirement associations and attenuated the accommodation association. The main limitations were the cross-sectional design, which precludes causal inference; the restriction of enrollment analyses to trials reporting baseline demographics, which were more recent than those without such reporting; and possible misclassification of automatically classified exposures. CONCLUSIONS: English-language proficiency requirements became more than three times as common between 2010 and 2023 and were associated with lower enrolled proportions of Asian and Hispanic/Latinx participants. Less common documented language accommodation was associated with a higher enrolled proportion of Hispanic/Latinx participants. Because these criteria are investigator-written and publicly recorded, they may be a tractable target for improving enrollment equity, although whether revising them changes who enrolls requires prospective evaluation

Purpose: We introduce the Temporal Index of Divergent Evidence (TIDE), a metric that evaluates how the topic center of gravity of a corresponding corpus of literature aligns with, or shifts relative to, a recommendation's semantics over time. Methods: TIDE computes annual mean cosine similarity using MedCPT embedding vectors between a recommendation and corresponding PubMed records binned by year (2000-2024), testing for monotonic temporal trend (Kendall {tau}). Three recommendations with expected divergent, stable, and convergent trajectories were examined: two-stage exchange for periprosthetic joint infection (PJI), emergent fasciotomy for acute compartment syndrome (ACS), and non-operative management of distal radius fractures (DRF) in adults [&ge;] 65 years. The signal was characterized by an additive per-token attribution, an isolated vs. in-context analysis, and an adversarial minimal-pair test. Results: PJI diverged ({tau} = -0.84, year-shuffle p = 0.001), DRF converged ({tau} = +0.42, p = 0.005), and ACS showed no detectable trend ({tau} = -0.07, p = 0.66). The two significant directions (PJI, DRF) held under matched input [CLS] pooling, while ACS remained near zero. Publication volume baselines did not distinguish the three trajectories (publication count was universally positive, {tau} = +0.77 to +0.97). A reversed-meaning minimal-pair control showed little cosine sensitivity to reversal. Paper-level TF-IDF recovered the qualitative three-case pattern. Conclusion: Embedding-based drift analysis is a proof-of-concept candidate pre-triage signal that tracks topical alignment, not evidentiary direction; superiority over TF-IDF remains unproven. The present study is hypothesis-generating, showing potential use of embedding representations as an analytical unit rather than simply for retrieval.

The Elixhauser comorbidity measures are among the most widely used risk adjusters in administrative health data. Their reference implementation, published by the Agency for Healthcare Research and Quality (AHRQ) as SAS programs, has eleven annual releases in two families, one screening pre-existing conditions on the present-on-admission (POA) indicator and the earlier one on Medicare Severity Diagnosis-Related Group, and no implementation of every release existed outside SAS. We developed `ecsr10`, an open-source R package covering all eleven releases of both families, with a browser-based application over the same functions, and validated it against AHRQ's own SAS programs over 40,027,114 value-level comparisons with zero disagreements. Two existing open-source reimplementations scored on the same dataset showed reproducible defects. We then used the package to quantify two choices AHRQ's software leaves to the analyst and studies seldom report, the release and the handling of diagnoses not present on admission, on the Texas Inpatient Public Use Data File, 2016 Q1-2019 Q4 (8,585,244 adult discharges from 726 hospitals), scoring one predefined cohort under every release and pre-existing-condition setting. POA handling dominated: disabling POA changed the comorbidity profile of 56.7% of admissions, and a POA-naive mortality index scored higher on discrimination (area under the curve 0.807 versus 0.789; difference 0.0183, 95% confidence interval 0.0161-0.0204) by counting in-hospital complications as pre-existing disease. POA reporting was bimodal across hospitals, so pooled hospital comparisons on such a file partly compare documentation practice. Because the cohort predates every release compared, the release contrast is a lower bound: the release chosen between v2022.1 and v2026.1 changed at least one comorbidity flag for 0.0036% of admissions, yet a single revision of ten mortality weights changed the comorbidity index for 42.4% of them. An exactly validated implementation makes such choices measurable; release, POA handling and the implementation used should be reported as study characteristics.

Selection bias is a common concern in causal inference. Two mechanisms can give rise to selection bias: collider bias or stratifying on an effect modifier of the exposure-outcome causal effect. Research into the impact of selection bias on instrumental variable (IV) analyses has focused primarily on collider bias. Here, we show that IV estimates can be biased if selection is an effect modifier (or caused by an effect modifier) of the exposure-outcome effect. In a simulation study, we investigate the size and direction of the bias when estimating effects on the mean difference or odds ratio scale. Bias can also occur due to heterogeneity in the instrument-exposure relationship, although its magnitude is typically smaller. Finally, we discuss methods to deal with this bias in practice, including inverse probability weighting (IPW).

Objective: To develop and evaluate a dynamic model that repeatedly estimates the 30-day risk of a first grade [&ge;]2 immune-related adverse event (irAE) during immune checkpoint inhibitor (ICI) therapy. Materials and Methods: Adults initiating ICI therapy during 2015-2022 comprised the development cohort; 2023 initiators formed a nonoverlapping internal temporal evaluation cohort. Every 21 days during the first treatment year, models used available EHR information to predict a first structured phenotype-defined above Grade 2 irAE during the next 30 days. Evaluation included pooled and within-landmark discrimination, precision-recall performance, calibration, and patient-clustered bootstrap confidence intervals. Feature-state ablations and alternative update schedules examined the contribution of dynamic information. Results: Development included 1,522 patients and 18,744 evaluable landmarks; temporal evaluation included 526 patients and 6,263 landmarks, of which 184 were positive (2.94%). The multiscale XGBoost model achieved pooled AUROC 0.716 (95% CI, 0.676-0.755), AUPRC 0.067, within-landmark AUROC 0.672 (95% CI, 0.631-0.716), Brier score 0.0281, and log loss 0.1245. The time-aware baseline achieved AUROC 0.631 and within-landmark AUROC 0.536. Updated treatment exposure and laboratory trajectories accounted for most of the gain. A complete-state extension achieved AUROC 0.719 and AUPRC 0.073. The 21-day schedule had the highest discrimination among tested cadences. Conclusion: Longitudinal EHR updating improved near-term irAE prediction beyond pretreatment characteristics and treatment time. Phenotype adjudication and independent evaluation of calibration, transportability, and clinical utility are needed before implementation

Background: Excess mortality estimates depend on an expected baseline, an unobservable quantity typically derived from models with prescribed functional forms and short reference periods that exclude exceptional events. Singular Spectrum Analysis (SSA) offers a nonparametric alternative but requires subjective component selection and is sensitive to outliers. We combined SSA with automated component selection and outlier interpolation to estimate long-term baselines and compile a catalog of excess mortality events (EMEs). Methods: We analyzed weekly all-cause mortality (2015-2024) for six metropolitan areas with contrasting climatic, demographic, and epidemiological contexts: Brussels, Antwerp, and Liege (Belgium), and Athens, Thessaloniki, and Larisa (Greece). SSA was applied to standardized log-mortality rates; components were selected using Monte Carlo SSA, and COVID-19 periods were interpolated iteratively. EMEs were defined as weeks exceeding a quasi-Poisson two-z-score threshold, then screened for coincidence with heat waves, cold waves, COVID-19 periods, and influenza epidemics. Results: Baselines were consistent within countries: annual cycles in Belgium, with an additional significant semiannual cycle in Greece reflecting summer mortality. We cataloged 170 EMEs spanning 456 weeks and 14,210 excess deaths. Pandemic weeks accounted for 77% of excess mortality, and 96 weeks coincided with temperature extremes; 56 EMEs, representing 3.7% of excess mortality, remained unmatched. Conclusions: SSA yields reproducible, low-assumption baselines that capture country-specific seasonality. Because flexible baselines can absorb recurrent seasonal mortality, component selection is a substantive choice that should be reported explicitly, together with its sensitivity.

Background Large language models (LLMs) show promise in scientific literature reviews, but cloud-based proprietary models like GPT-4 raise concerns around privacy, copyright, and licensed article access. Using a scoping review on AI applications for treatment effect estimation as a use case, this study evaluated local open-source LLMs for automated article screening in scientific literature reviews and developed a practical workflow for their implementation. Methods We extracted a sample dataset of 300 records from an ongoing scoping review on machine learning methods for treatment effect estimation using real-world medical data. We divided the dataset into training (n = 200) and testing (n = 100) datasets. We evaluated 11 locally deployed open-source LLMs from five model families (GPT-OSS, Gemma, Qwen, Mistral, Llama) for both abstract screening and full-text screening. Full-text screening was done using text converted by two PDF-to-text conversion tools: Docling and PyMuPDF4LLM. Performance was assessed using raw accuracy with 95% Wilson's confidence interval, relaxed accuracy, Cohen's kappa, and runtime. During workflow development, prompt engineering, output extraction strategies, and generation settings were iteratively optimized using the training dataset. Results GPT-OSS-20B achieved the highest raw accuracy for title and abstract screening (0.890 on the training set and 0.850 on the test set), followed closely by Gemma-4-31B-it (0.875 on the training set and 0.830 on the test set) and Gemma-4-26B-A4B-it (0.810 on the training set and 0.790 on the test set). In full-text screening, slightly higher raw accuracy and reduced false-positive rates were observed, with GPT-OSS-20B and Gemma-4 variants consistently demonstrating strong performance. Only minor variations were found between the two PDF-to-text conversion methods evaluated, though runtime varied substantially across models. Based on these findings, we compiled a set of practical recommendations for local LLM-assisted article screening. Conclusions Local LLMs can effectively support literature screening when integrated into a carefully designed workflow. Recent generations of open-source local models achieved performance comparable to human reviewers and proprietary LLMs. Local deployment also provides advantages in data privacy, copyright compliance, customization, and cost-efficiency. While human oversight remains essential, the proposed workflow demonstrated that local LLMs represent a practical and reproducible approach for accelerating evidence synthesis.

Biomedical Machine Learning (ML) is increasingly evaluated through Independent and Identically Distributed (IID) benchmarks, even when deployment involves irregular observation, missingness, distribution shift, and high failure costs. This position paper argues for a stability-oriented reporting standard for safety-relevant biomedical ML. The central requirement is not a specific model class or optimizer, but auditable evidence: performance should be reported under clinically plausible perturbations such as thinning, timestamp jitter, bursty missingness, channel dropout, and Signal-to-Noise Ratio (SNR) shift. When authors invoke training-stability diagnostics, those diagnostics should be hazard-linked: their assumptions, operational meaning, and relationship to stress-test degradation should be stated explicitly. We use controlled mechanism probes to illustrate two hazards that IID scores can hide: observation-process dependence under irregular sampling, and perturbation sensitivity associated with high-curvature training regimes such as Edge of Stability (EoS). We do not propose Neural ODEs, continuous-time models, Sharpness-Aware Minimization (SAM), or EoS metrics as default prescriptions. Instead, we propose a minimum reporting standard that makes robustness claims testable and supports escalation to more complex modelling or optimization only when stress tests justify the cost.

In systematic review, the measures reported by the primary studies may not always be the measures sought by the systematic review investigators. Motivated by an systematic review of interventions to reduce meat consumption, we discuss here methods to infer relative risks and confidence intervals from randomized controlled trials that reported differences, possibly in the presence of repeated measures or multiple interventions.

ImportanceCardiovascular disease remains the leading cause of mortality worldwide. Although machine learning models have demonstrated strong predictive performance, limited attention has been given to uncertainty quantification and gender-specific risk factor identification, which are essential for clinically trustworthy artificial intelligence. ObjectiveTo develop and evaluate an interpretable and uncertainty-aware machine learning framework for cardiovascular disease prediction that integrates an association study on biomarkers, explainable artificial intelligence, and gender-stratified risk assessment. Design, Setting, and ParticipantsThis retrospective observational study utilized the UCI Heart Disease dataset comprising 918 participants, including 725 males and 193 females. Demographic, clinical, laboratory, and exercise-related variables were analyzed. Missing values were addressed using a hybrid imputation strategy combining MICE and supervised Random Forest imputation. MethodsAn ensemble machine learning imputation strategy was developed to address missingness. Then advanced machine learning models were developed and evaluated using stratified training/testing partitions and 5-fold cross-validation. Predictive performance was assessed using several evaluation metrics. Uncertainty was quantified through bootstrap resampling, predictive entropy, and mutual information. Linear Double machine learning and Causal Forest double machine learning were applied to estimate gender-specific averages and association studies for risk factors. ResultsRandom Forest achieved the strongest and most stable predictive performance across male and female cohorts, with ROC-AUC values exceeding 0.90 and robust cross-validation performance. Hybrid imputation improved classification performance, increasing the F1-score from 0.87 to 0.89 and ROC-AUC from 0.90 to 0.92. The causal association study analyzes identified the number of major vessels, ST-segment slope, thalassemia, and exercise-induced angina as the strongest determinants of heart disease, with notable differences observed between male and female subgroups. Conclusions and RelevanceIntegrating predictive modeling, uncertainty quantification, and association study machine learning provides a robust and clinically interpretable framework for cardiovascular disease risk assessment. The proposed approach improves prediction reliability while identifying gender-specific causally important risk factors, supporting precision cardiovascular medicine and trustworthy clinical artificial intelligence.

TrialCalibre1 is an agentic system that connects clinical evidence, study design, and executable analyses. It supports reference-trial replication, direct target-trial emulation, and BenchExCal, which combines benchmarking, extension, and cal ibration sensitivity analysis. We evaluated an ASCET-based benchmark and an extension comparing clopidogrel with aspirin among adults aged 81-90 years us ing synthetic PCORnet-style claims. Two reviewers assessed benchmark extrac tion, protocol decisions, and implementation. After discussion, 53/62 and 50/62 assessable SQL checks were rated met; both reviewers rated 12/13 assessable R checks met. Protocol and proxy disagreements remained. A separate ten-question routing development set yielded 10/10 route matches and 7/7 applicable reference trial matches. The agent benchmark and extension included 6,069 and 1,602 pa tients. The extension risk ratio was 0.848 (95% CI 0.569-1.184), with a calibrated sensitivity estimate of 1.274 (0.439-3.700). The human implementation provided descriptive comparisons, not same-data validation. Dose proxies, unpopulated co variates, residual imbalance, implementation deviations, and ignored cross-cohort covariance limited interpretation. This case demonstrates a connected workflow whose protocols, code, and results can be reviewed.

Public data sharing in immunological and clinical research is essential for transparency, reproducibility, and secondary analysis. However, many datasets in repositories such as ImmPort lack sufficient structured data to enable reuse. ImmPort's Assessments and Lab Tests templates -- which capture critical clinical information -- are often incomplete or omitted due to their complexity and the manual effort required. To address this challenge, an application with a graphical user interface requiring no programming was developed to facilitate population of these templates. The tool streamlines the submission process by reducing time and complexity for data contributors while ensuring template formatting. It integrates ImmPort files, study-specific data files, and a curated data dictionary to automate the mapping of study variables to the required fields within each template. The tool was evaluated on seven ImmPort studies with diverse designs. It successfully generated submission-ready templates that passed ImmPort's internal validation and will be incorporated into the repository. By improving the feasibility and completeness of templates, this approach enhances the quality, findability, and reusability of data within ImmPort.

This paper shows that it is possible to predict which newly published biomedical research papers will be connected to subsequent clinical trials. We use CAR-T cell therapy as a proof-of-concept setting because its development can be followed from published research through clinical testing, regulatory approval, and commercialization. We reconstructed connections between research papers and clinical trials and trained machine-learning models to predict which papers were associated with research that progressed to clinical testing. Using 65 paper-inherent indicators, including scientific topic, language, biomedical orientation, and team structure, the models achieved a cross-validated AUC of 0.908. The resulting rankings were also practically informative. In a retrospective analysis using no information generated after 2016, 80% of the top 20 papers came from laboratories associated with one of 13 predefined major CAR-T clinical or commercial outcomes. As an additional finding, first-year citations alone achieved an AUC of 0.797, while provenance indicators based on author, journal, and institutional reputation performed less strongly and added little to the paper-inherent model. Our results show that whether biomedical research papers will be connected to subsequent clinical trials can be predicted from newly published papers with high discrimination in CAR-T. Our findings could help research organizations and pharmaceutical companies identify papers, laboratories, and therapeutic programs that warrant closer evaluation before clinical outcomes are known.

Large language models (LLMs) screen titles and abstracts without review-specific training, but generating screening decisions as text takes processing time and incurs API charges. We evaluated Jev, a non-generative model returning classification probabilities, on 4527 records from two systematic reviews of bipolar disorder treatments. We used human reviewers' decisions to retain records for full-text assessment as the reference standard. In the primary analysis, we asked Jev yes-or-no questions ("Noul") about whether to retain each record. Records were retained when their retention probability was at least 50%, a cutoff specified before the final evaluation runs. We also evaluated lower retention thresholds. For comparison, we gave Jev explicit include/exclude options ("Choice") and used the same prompts with GPT-5 mini and GPT-6 Astra; both LLMs used default reasoning. In the primary analysis at 50%, sensitivity and specificity were 99.6% and 98.9% for light therapy and 93.2% and 93.5% for adjunctive pharmacotherapy. Lower cutoffs selected on these data retained all reference-positive records. Compared with assessing every record manually, these cutoffs reduced the number requiring human assessment by 96.9% for light therapy and 61.3% for adjunctive pharmacotherapy. For light therapy and adjunctive pharmacotherapy, sensitivity was 92.3% and 90.0% with Jev using explicit options at 50%, 87.2% and 90.0% with GPT-5 mini, and 92.3% and 80.8% with GPT-6 Astra. Costs for screening all 4527 records once were US$0.20 for Jev using explicit options, US$2.59 for GPT-5 mini, and US$33.45 for GPT-6 Astra. Jev's probabilities could provide graded information for reviewers assessing titles and abstracts.